Augmented reality (ar) display of pipe inspection data

ABSTRACT

Described is a method of providing an augmented reality (AR) scene of pipe inspection data, including: obtaining, using a processor, pipe inspection data derived from a pipe inspection robot that traverses through the interior of an underground pipe, the pipe inspection data including one or more sets of condition assessment data relating to an interior of the underground pipe; obtaining, using a processor, real-time visual image data of an above-ground surface; combining, using a processor, the pipe inspection data with the real-time visual image data in an AR scene; and displaying, using a display device, the AR scene. Other examples are described and claimed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. provisional patent applicationSer. No. 62/624,838, entitled “AUGMENTED REALITY (AR) DISPLAY OF PIPEINSPECTION DATA” and filed on 1 Feb. 2018, the contents of which areincorporated by reference herein.

BACKGROUND

Pipes that carry water, other fluids and gases are an important type ofinfrastructure. A great deal of pipe data is captured in still images orvideo, e.g., using cameras to record information from the visiblespectrum of light. However, other data can provide additionalinformation beyond what is visible to the naked eye. For example,acoustic, ultraviolet (UV), laser, and infrared (IR) imaging have beenutilized to identify details related to pipe topology or condition.

Various systems exist that create pipe inspection data, for exampleobtained via a pipe inspection robot, in a variety of formats.Conventionally pipe inspection data is presented in a two-dimensional(2D) format as either still image data or video data. Some systems arecapable of presenting three-dimensional (3D) information in the form of3D-like graphics that are presented on a flat (2D) screen.

BRIEF SUMMARY

In summary, one embodiment provides a method of providing an augmentedreality (AR) scene of pipe inspection data, comprising: obtaining, usinga processor, pipe inspection data derived from a pipe inspection robotthat traverses through the interior of an underground pipe, the pipeinspection data including one or more sets of condition assessment datarelating to an interior of the underground pipe; obtaining, using aprocessor, real-time visual image data of an above-ground surface;combining, using a processor, the pipe inspection data with thereal-time visual image data in an AR scene; and displaying, using adisplay device, the AR scene.

Another embodiment provides a device, comprising: a display device; acamera; a processor; and a memory that stores processor executableinstructions comprising: instructions that obtain, using the processor,pipe inspection data derived from a pipe inspection robot that traversesthrough the interior of an underground pipe, the pipe inspection dataincluding one or more sets of condition assessment data relating to aninterior of the underground pipe; instructions that obtain, using thecamera, real-time visual image data of an above-ground surface;instructions that combine, using the processor, the pipe inspection datawith the real-time visual image data in an AR scene; and instructionsthat display, using the display device, the AR scene.

A further embodiment provides a computer program product, comprising: anon-transitory storage medium that stores processor executableinstructions, comprising: instructions that obtain pipe inspection dataderived from a pipe inspection robot that traverses through the interiorof an underground pipe, the pipe inspection data including one or moresets of condition assessment data relating to an interior of theunderground pipe; instructions that obtain real-time visual image dataof an above-ground surface; instructions that combine the pipeinspection data with the real-time visual image data in an AR scene; andinstructions that display, using the display device, the AR scene.

The foregoing is a summary and thus may contain simplifications,generalizations, and omissions of detail; consequently, those skilled inthe art will appreciate that the summary is illustrative only and is notintended to be in any way limiting.

For a better understanding of the embodiments, together with other andfurther features and advantages thereof, reference is made to thefollowing description, in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates an example method of providing pipe inspection datato an augmented reality view.

FIG. 2(A-B) illustrates example views of an augmented reality display ofpipe inspection data.

FIG. 3 (A-C) illustrates example views of controlling an augmentedreality display of pipe inspection data.

FIG. 4(A-C) illustrates example views of controlling an augmentedreality display of pipe inspection data.

FIG. 5 illustrates an example augmented reality display of pipeinspection data.

FIG. 6 illustrates an example augmented reality display of pipeinspection data.

FIG. 7 illustrates an example system.

FIG. 8 illustrates an example of augmented reality display of pipeinspection data.

DETAILED DESCRIPTION

It will be readily understood that the components of the embodiments, asgenerally described and illustrated in the figures herein, may bearranged and designed in a wide variety of different configurations inaddition to the described example embodiments. Thus, the following moredetailed description of the example embodiments, as represented in thefigures, is not intended to limit the scope of the claims, but is merelyrepresentative of those embodiments.

Even if a pipe inspection robot is utilized, the resultant data producedby conventional systems is often difficult for the end user to grasp.The pipe inspection data may be processed to relate 2D and 3Dinformation of the pipe's interior; however, this data is oftendifficult to interpret visually in 2D display formats. Moreover, for agiven pipe segment, although its inspection data may be relevant andunderstood by the end user, its place or overall context within the pipenetwork may be difficult to grasp, as some pipe networks are quiteextensive. These technical issues present problems for end users thatneed to make decisions regarding the pipe network, e.g., city managersthat must decide whether to expend resources rehabilitating or replacingparticular segments of pipe within a pipe network.

Accordingly, an embodiment provides methods, devices and products formore effectively visualizing the interior of a pipeline by using anaugmented reality (AR) system. The AR system combines virtualrepresentations, e.g., 3D pipe LIDAR (light detecting and ranging) scandata, with visual images, e.g., the above ground view of the pipelocation. Users may therefore gain a better perspective for thecondition of a pipe segment and where it is located within a pipenetwork by utilizing an AR system.

In an embodiment, an AR scene of pipe inspection data is provided byobtaining, using a processor, pipe inspection data derived from a pipeinspection robot that traverses through the interior of an undergroundpipe. The pipe inspection data includes one or more sets of conditionassessment data, for example laser scan data, sonar scan data, andvisual image data, relating to an interior of the underground pipe. Aprocessor obtains real-time visual image data of an above-groundsurface, e.g., from a camera included in a mobile device such as atablet, smartphone or heads up or head mounted display. A processorcombines the pipe inspection data, which may be in the form of acylindrical, three-dimensional (3D) representation of the pipe formedusing laser scan data, with the real-time visual image data into an ARscene. The AR scene is then displayed using a display device, e.g.,included in the mobile device.

In an embodiment, a user input is detected and the display of the ARscene is updated in response to the user input. The user input mayinclude one or more of movement of the display device (e.g., panning,tilting or zooming the mobile device display, as detected via anaccelerometer, gyroscope or other sensor(s)), gesture input and voiceinput. The updating of the display of the AR scene includesrepositioning one or more of the pipe inspection data (e.g., a 3D scanimage of the pipe) and the real-time visual image data within the ARscene. The updating of the display of the AR scene may includerepositioning the pipe inspection data, e.g., relative to the real-timevisual image data, in response to user input selected from the groupconsisting of gesture input and voice input. This allows, for example,the pipe inspection data to be independently repositioned, e.g., raisedup with respect to the image of the surface of the ground, ortransformed, e.g., transitioned from a 3D pipe representation to aplanar or flat graph representation.

In an embodiment, the pipe inspection data includes data relating to anin-ground position of the underground pipe and the real-time visualimage data includes surface feature data indicating a position of asurface feature. In an embodiment, the combining of the real-time visualimages and the pipe inspection data includes using the data relating tothe in-ground position of the underground pipe to align at least a partof the pipe inspection data with the surface feature in the real-timevisual image data. For example, computer vision techniques such aspattern matching, object identification, and the like can be used todetect the surface feature in the real-time visual image and anembodiment can detect a related in-ground position feature in the pipeinspection data to align at least a part of the pipe inspection data,e.g., an opening for a manhole cover, with the surface featureidentified in the real-time visual image data, e.g., the correspondingphysical manhole cover. Other position data may be used to assist thisprocess or in lieu of computer vision techniques. For example, GPS orother location data or orientation data such as compass data may be usedto determine the location and direction of the view offered by real-timevisual images and position the pipe inspection data in the AR scene.

The description now turns to the figures. The illustrated exampleembodiments will be best understood by reference to the figures. Thefollowing description is intended only by way of example and simplyillustrates certain selected example embodiments.

Referring to FIG. 1, as illustrated at 101 an embodiment obtains pipeinspection data, e.g., visual image data, laser scan data, etc., of asegment of a pipe, collected as the pipe inspection robot traversesthrough the interior of the pipe. The pipe inspection data may becollected by at least one sensor or an array of sensors mounted to amobile inspection robot, e.g., LIDAR units, sonar units, and a visualcamera.

At 102 an embodiment processes the collected pipe inspection data toformat it appropriately for AR display. By way of example, a laser scanimage of the interior of the pipe may be formed into a cylindricallength of pipe. Colors or other visual data may indicate the internalcontours of the pipe's interior surface, e.g., sediment buildup orerosion/cracks in the pipe wall.

An embodiment combines the processed pipe inspection data withcontextual data at 103, e.g., combined with visual images of the surfaceof the ground, to form an AR scene or view of a pipe segment in context,e.g., as it sits in the ground beneath a surface location. Thiscombination with contextual data may take place in real-time (whichincludes “near” real-time, as will be understood by those havingordinary skill in the art). For example, the combination may occur asthe pipe inspection robot traverses through the interior of the pipe, orlater. Further, the contextual data may be obtained in real time, e.g.,as the user walks along the surface above the pipe. This composite imageis output at 104 to an appropriate display device, permitting a user toview the pipe scan data in a new context, e.g., through a head mounteddisplay, on a device screen pointed at the surface location, etc.

As illustrated at 105, if a user provides an input such as a gesturedetected by a head mounted display, the AR display may be modified,e.g., moving the pipe scan data/virtual image relative to the real,visual image. Otherwise, the current AR display or scene may bemaintained, as illustrated at 106.

By way of example, an association may be made, e.g., based on apredetermined or coded rule, between a gesture detected from the userand a modification of the AR scene. By way of specific example, apredetermined gesture such as raising a hand may result in apredetermined scene change, e.g., moving the pipe scan virtual imagerelative to the visual image to lift the pipe virtually out of theground for above surface viewing. This permits the user to manipulatethe AR scene, as illustrated at 107 and 108.

Illustrated in FIG. 2A-B are example views of an AR scene 200 a, 200 bproduced by an embodiment. As shown in FIG. 2A, an embodiment providesan AR scene 200 a that combines virtual images of a virtual manholecover 201 a and laser pipe scan data 202 a with visual images of thesurface of the ground 203 a. As illustrated, the virtual manhole cover201 a is situated above the pipe scan data 202 a in the AR scene 200 a.Thus, the distance between the pipe scan data 202 a and the virtualmanhole cover 201 a in the AR scene 200 a represents the distancebetween the physical pipe and the physical manhole cover. In anembodiment, the scale or distance between a virtual manhole cover 201 aand pipe scan data 202 a (or other features) may be changed.

In an embodiment, the virtual features, e.g., features 201 a, 202 a, maybe aligned or oriented with features of the visual image 203 a. Thisalignment of features corresponds to the actual orientation of physicalelements, such as an underground pipe, to which the virtual featurescorrespond. For example, the position of the physical pipe as it sitsbeneath the ground is mimicked in the AR scene 200 a by aligning thepipe scan data 202 a within the AR scene 200 a appropriately.

In an embodiment, features within pipe scan data 202 a and visual imagedata 203 a are used to align the elements within the AR scene 200 a.That is, feature locations within the pipe scan data 202 a are alignedwith surface features in the visual image 203 a so as to present an ARscene 200 a that represents the pipe's actual (or scaled) locationwithin ground and aligned appropriately within the visual image 203 a.The pipe's location within the ground may be obtained or aligned, forexample, using marker data found within the pipe scan data 202 a, e.g.,data indicating a location of a feature in the pipe such as an openingfor a manhole cover or a marker within the pipe of a manhole position.

The features (surface or subterranean) may be detected and identifiedautomatically, e.g., using computer vision or pattern matchingtechniques to identify such features. In an embodiment, a pipeinspection robot may be configured to automatically detect such featuresin pipe inspection data (e.g., laser scan data). An embodiment maylikewise automatically detect surface features in the visual image data(e.g., real-time visual images of the surface of the ground) and relatethese to features in the pipe inspection data.

Having obtained feature(s) from the pipe inspection data, the feature(s)within the visual images are related thereto for use in building the ARscene 200 a. For example, to align the pipe scan data 202 a within theAR scene 200 a, an embodiment may automatically detect a physicalfeature in an underground pipe. The feature may be a feature that can bescanned such as a hole in the interior of the pipe that is sized and/orlocated in a pipe segment where a manhole opening is expected, a markerof a manhole entry, etc. Similarly, an embodiment may detect a surfacefeature that is visually scanned or identified from a visual image 203a, such as labeling on a manhole cover. An embodiment automaticallyidentifies these features, e.g., using matching to known features. In anembodiment, a physical surface feature of at least 2 inches in size isused to facilitate such identification using computer vision techniques.Features in the pipe scan data 202 a are then related to features withinthe visual image 203 a, e.g., using rules or machine learning. Forexample, a hole within a pipe scan 202 a may be aligned with a physicalmanhole location identified in a visual image and scaled to appear belowit in the AR scene 200 a. Depending on the state of the AR scene 200 a,the pipe scan data 202 a may be located a predetermined distance belowthe location of the physical manhole identified in the visual image 203a to mimic the pipe's in-ground location.

As illustrated in FIG. 2B, a user may reposition the viewing device,e.g., a head set, a mobile device such as a smart phone or tablet, etc.,to view other areas of the pipe scan data. For example, the view in FIG.2B is the result of a user panning the device to view to the left withrespect to FIG. 2A, e.g., by moving the AR headset to the left to viewthe left side of the pipe scan data 200 a. As such, the user can viewthe updated AR scene 200 b, including the left side of the pipe scandata 202 b, another area of the surface of the ground 203 b, and adifferent virtual manhole cover 204 b. In this case, the user isproviding inputs via device movement, detected using device sensors suchas an accelerometer.

FIG. 3A-C illustrates successive views where an AR scene is updated byother user input, e.g., a hand gesture viewed by a camera of a headmounted display. In FIG. 3A-C, the right end of the pipe scan image (302a-c) is closest to the user. In FIG. 3A, the AR scene includes virtualmanhole 301 a, pipe scan data or image 302 a, surface/visual imagery 303a, and virtual manhole 304 a, in a first orientation. As can beappreciated, the visual images are captured in real time and combinedwith the pipe scan data (which may have been obtained at an earliertime). The user's hand is visible in a first orientation 305 a.

In FIG. 3B, the user has performed a pinch gesture 306 b, which changesor updates the AR scene. In this example, the pinch gesture 306 b isassociated with a coded rule that lifts or raises the pipe scan data 302b relative to the ground imagery 303 b and the virtual manholes 301 b,304 b.

As shown in FIG. 3C, the pipe scan data 302 c lifts or raises itselfabove the virtual manhole 304 c and relative to the ground imagery 303c. This permits the user to view the pipe scan data 302 c from adifferent angle, e.g., the user can lift the pipe scan data 302 c intothe air for a closer look or to view the underside of the pipe scan data302 c.

FIG. 4A-C illustrates another example modification of an AR scene. InFIG. 4A-C, the user has moved (e.g., walked) nearer to the right end ofthe pipe scan image (402 a-c), making this end closer to the user (i.e.,the user is looking down the length of the pipe scan image 402(a-c) fromthe right end to the left end). In this example, after the user haslifted or raised the pipe scan data 402 a with respect to visual imagery403 a, a second user pinch gesture 407 a begins an animation of the pipescan data 402 a unraveling to reveal a flat graph view 402 c of thepipe's interior (as illustrated by the sequence of FIG. 4B-C).Therefore, the pipe scan data transitions through views 402 b, 402 c,with respect to the visual imagery 403 b, 403 c in the AR scene.

FIG. 5 shows an example view of a resultant flat graph view of the pipescan data 502 that results from the operation performed in FIG. 4A-C. Asshown, the user has moved or panned to another location on the surfaceof the ground to look at the pipe scan data 502, e.g., the user haswaked down the left end of the pipe scan data to view the pipe scan datafrom the other end, from the right end to the left end. As illustrated,the view of the flat graph 502 is maintained relative to the surfaceimagery 503, which includes physical manhole cover 508 (corresponding tovirtual manhole cover 504).

In an embodiment, the pipe scan data is maintained in an actual orrelative location with respect to the visual imagery using locationmarkers and/or GPS data. For example, in FIG. 6 the pipe scan data 602is aligned with a virtual manhole cover 604 as well as the actualmanhole cover 608. This permits a user to view the pipe scan data 602 incontext, i.e., where the pipe actually sits (or relative thereto, e.g.,raised by gesture or another user input). Again, a relative marker maybe identified using computer vision techniques (e.g., matching letteringon a manhole cover) and relating a characteristic in the visual image,e.g., location of a detected manhole cover in visual image, withlocation of the end of the pipe scan data 602 and virtual manhole cover604, forming a contextually accurate AR scene.

In an embodiment, the pipe scan data can be combined with detailedvisual imagery to provide a contextual view, e.g., a pipe scan datalocation within a city street, as illustrated in FIG. 8. As illustrated,pipe scan data 802 and a virtual manhole cover 801 are aligned with oneanother, as described herein. In the example of FIG. 8, the virtualmanhole cover 801 is situated (virtually) above the opening 809 in thepipe scan data 802. Such features as virtual manhole cover 801 andopening 809 permit an embodiment to align the pipe scan data within anAR scene that includes visual images. For example, these features 801,809 permit an embodiment to align the pipe scan data 802 with real worldfeatures such as an above ground manhole cover within the AR scene. Thisalignment may be refined with additional data, e.g., GPS data, map data(e.g., of pipe segment locations located in a city street), etc.Furthermore, an embodiment may roughly align the pipe scan data 802using certain data, e.g., GPS data, map data, etc., rather than aligningvirtual and visual feature using computer vision, which may beappropriate in certain circumstances such as when a precise location ofthe pipe scan data is not needed or desired.

It will be readily understood that certain embodiments can beimplemented using any of a wide variety of devices or combinations ofdevices. Referring to FIG. 7, an example system on chip (SoC) includedin a computer 700 is illustrated, which may be used in implementing oneor more embodiments. The SoC or similar circuitry outlined in FIG. 7 maybe implemented in a variety of devices in addition to the computer 700,for example similar circuitry may be included in a pipe inspection robot770, another device platform, and/or an AR system 770 a. In addition,circuitry other than a SoC, an example of which is provided in FIG. 7,may be utilized in one or more embodiments. The SoC of FIG. 7 includesfunctional blocks, as illustrated, integrated onto a singlesemiconductor chip to meet specific application requirements.

The central processing unit (CPU) 710, which may include one or moregraphics processing units (GPUs) and/or micro-processing units (MPUs),includes an arithmetic logic unit (ALU) that performs arithmetic andlogic operations, instruction decoder that decodes instructions andprovides information to a timing and control unit, as well as registersfor temporary data storage. The CPU 710 may comprise a single integratedcircuit comprising several units, the design and arrangement of whichvary according to the architecture chosen.

Computer 700 also includes a memory controller 740, e.g., comprising adirect memory access (DMA) controller to transfer data between memory750 and hardware peripherals. Memory controller 740 includes a memorymanagement unit (MMU) that functions to handle cache control, memoryprotection, and virtual memory. Computer 700 may include controllers forcommunication using various communication protocols (e.g., I2C, USB,etc.).

Memory 750 may include a variety of memory types, volatile andnonvolatile, e.g., read only memory (ROM), random access memory (RAM),electrically erasable programmable read only memory (EEPROM), Flashmemory, and cache memory. Memory 750 may include embedded programs anddownloaded software, e.g., image processing software, pipe inspectionmission software, etc. By way of example, and not limitation, memory 750may also include an operating system, application programs, otherprogram modules, and program data.

A system bus permits communication between various components of thecomputer 700. I/O interfaces 730 and radio frequency (RF) devices 720,e.g., WIFI and telecommunication radios, are included to permit computer700 to send and receive data to remote devices using wired or wirelessmechanisms. The computer 700 may operate in a networked or distributedenvironment using logical connections to one or more other remotecomputers or databases. The logical connections may include a network,such local area network (LAN) or a wide area network (WAN), but may alsoinclude other networks/buses. For example, computer 700 may communicatedata with and between a pipe inspection robot 770 and AR devices 770 a.

The computer 700 may therefore execute program instructions configuredto store and analyze pipe data, and perform other functionality of theembodiments, as described herein. A user can interface with (forexample, enter commands and information) the computer 700 through inputdevices, which may be connected to I/O interfaces 730. A display orother type of device, e.g., AR system 770 a, may also be connected tothe computer 700 via an interface selected from I/O interfaces 730, suchas an output interface.

It should be noted that the various functions described herein may beimplemented using instructions stored on a memory, e.g., memory 750,that are transmitted to and executed by a processor, e.g., CPU 710.Computer 700 includes one or more storage devices that persistentlystore programs and other data. A storage device, as used herein, is anon-transitory storage medium. Some additional examples of anon-transitory storage device or medium include, but are not limited to,storage integral to computer 700, such as a hard disk or a solid-statedrive, and removable storage, such as an optical disc or a memory stick.

Program code stored in a memory or storage device may be transmittedusing any appropriate transmission medium, including but not limited towireless, wireline, optical fiber cable, RF, or any suitable combinationof the foregoing.

Program code for carrying out operations may be written in anycombination of one or more programming languages. The program code mayexecute entirely on a single device, partly on a single device, as astand-alone software package, partly on single device and partly onanother device, or entirely on the other device. In some cases, thedevices may be connected through any type of connection or network,including a local area network (LAN) or a wide area network (WAN), orthe connection may be made through other devices (for example, throughthe Internet using an Internet Service Provider), through wirelessconnections or through a hard wire connection, such as over a USBconnection.

Pipe inspection robot(s) used for obtaining pipe scan data, asreferenced herein, may take a variety of forms and the example shown inFIG. 7 is a non-limiting example. The pipe inspection robot 770 of FIG.7 is an autonomous pipe inspection robot that includes a tractor module773 with tracks 774, 775, situated beneath a riser 772 that supports asensor module 771. Sensor module 771 comprises sensors, e.g., laser,sonar or visual sensors. Other pipe inspection robots may be used, e.g.,a raft or floating platform, a larger tracked platform, etc. In anembodiment, a plurality of pipe inspection robots may be used to obtainpipe inspection data of various kinds.

Example embodiments are described herein with reference to the figures,which illustrate example methods, devices and program products accordingto various example embodiments. It will be understood that the actionsand functionality may be implemented at least in part by programinstructions. These program instructions may be provided to a processorof a device to produce a special purpose machine, such that theinstructions, which execute via a processor of the device implement thefunctions/acts specified.

It is worth noting that while specific blocks are used in the figures,and a particular ordering of blocks has been illustrated, these arenon-limiting examples. In certain contexts, two or more blocks may becombined, a block may be split into two or more blocks, or certainblocks may be re-ordered or re-organized as appropriate, as the explicitillustrated examples are used only for descriptive purposes and are notto be construed as limiting.

Although illustrative example embodiments have been described hereinwith reference to the accompanying figures, it is to be understood thatthis description is not limiting and that various other changes andmodifications may be affected therein by one skilled in the art withoutdeparting from the scope or spirit of the disclosure.

What is claimed is:
 1. A method of providing an augmented reality (AR) scene of pipe inspection data, comprising: obtaining, using a processor, pipe inspection data derived from a pipe inspection robot that traverses through the interior of an underground pipe, the pipe inspection data including one or more sets of condition assessment data relating to an interior of the underground pipe; obtaining, using a processor, real-time visual image data of an above-ground surface; combining, using a processor, the pipe inspection data with the real-time visual image data in an AR scene; and displaying, using a display device, the AR scene.
 2. The method of claim 1, comprising detecting a user input and updating the display of the AR scene in response to the user input.
 3. The method of claim 2, wherein the user input includes one or more of movement of the display device, gesture input and voice input.
 4. The method of claim 2, wherein the updating the display of the AR scene comprises repositioning one or more of the pipe inspection data and the real-time visual image data within the AR scene.
 5. The method of claim 4, wherein the updating the display of the AR scene comprises repositioning the pipe inspection data in response to user input selected from the group consisting of gesture input and voice input.
 6. The method of claim 5, wherein the updating the display of the AR scene comprises transforming the pipe inspection data from a cylindrical representation of a pipe to a planar representation of a pipe.
 7. The method of claim 1, wherein: the pipe inspection data comprises data relating to an in-ground position of the underground pipe; and the real-time visual image data comprises surface feature data indicating a position of a surface feature.
 8. The method of claim 7, wherein the combining comprises using the data relating to the in-ground position of the underground pipe to align at least a part of the pipe inspection data with the surface feature in the real-time visual image data.
 9. The method of claim 8, comprising: using computer vision to detect the surface feature in the real-time visual image; and detecting a related in-ground position feature in the pipe inspection data to align the at least a part of the pipe inspection data with the surface feature in the real-time visual image data.
 10. The method of claim 9, wherein: the surface feature in the real-time visual image comprises at least a part of a physical man hole cover; and the related in-ground position feature in the pipe inspection data comprises a gap in the pipe inspection data associated with an opening in the underground pipe for a manhole.
 11. A device, comprising: a display device, a camera; a processor; and a memory that stores processor executable instructions comprising: instructions that obtain, using the processor, pipe inspection data derived from a pipe inspection robot that traverses through the interior of an underground pipe, the pipe inspection data including one or more sets of condition assessment data relating to an interior of the underground pipe; instructions that obtain, using the camera, real-time visual image data of an above-ground surface; instructions that combine, using the processor, the pipe inspection data with the real-time visual image data in an augmented reality (AR) scene; and instructions that display, using the display device, the AR scene.
 12. The device of claim 11, wherein the display device comprises a display device selected from the group consisting of a handheld two-dimensional display device and a head mounted display device.
 13. The device of claim 11, comprising instructions that detect a user input and instructions that update the display of the AR scene in response to the user input.
 14. The device of claim 13, wherein the user input includes one or more of movement of the display device, gesture input and voice input.
 15. The device of claim 13, wherein the instructions that update the display of the AR scene comprise instructions that reposition one or more of the pipe inspection data and the real-time visual image data within the AR scene.
 16. The device of claim 15, wherein the instructions that update comprise instructions that transform the pipe inspection data from a cylindrical representation of a pipe to a planar representation of a pipe.
 17. The device of claim 11, wherein: the pipe inspection data comprises data relating to an in-ground position of the underground pipe; and the real-time visual image data comprises surface feature data indicating a position of a surface feature.
 18. The device of claim 17, wherein the instructions that combine comprise instructions that use the data relating to the in-ground position of the underground pipe to align at least a part of the pipe inspection data with the surface feature in the real-time visual image data.
 19. The device of claim 18, comprising: instructions that use computer vision to detect the surface feature in the real-time visual image; instructions that detect a related in-ground position feature in the pipe inspection data to align the at least a part of the pipe inspection data with the surface feature in the real-time visual image data; wherein: the surface feature in the real-time visual image comprises at least a part of a physical man hole cover; and the related in-ground position feature in the pipe inspection data comprises a gap in the pipe inspection data associated with an opening in the underground pipe for a manhole.
 20. A computer program product, comprising: a non-transitory storage medium that stores processor executable instructions, comprising: instructions that obtain pipe inspection data derived from a pipe inspection robot that traverses through the interior of an underground pipe, the pipe inspection data including one or more sets of condition assessment data relating to an interior of the underground pipe; instructions that obtain real-time visual image data of an above-ground surface; instructions that combine the pipe inspection data with the real-time visual image data in an augmented reality (AR) scene; and instructions that display, using the display device, the AR scene. 